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Winograd  '(  has  given  a  new  method  for  computing  inner  products  . 
Under  certain  circumstances,  when  a  series  of  inner  products  must  be 
calculated,  using  Winograd 's  scheme  is  more  efficient  than  the  standard 
(naive)  method.  Winograd  points  out  that  his  method  does  matrix  multi¬ 
plication  up  to  twice  as  fast  as  the  usual  scheme  and  notes  similar 
acceleration  for  matrix  inversion  and  the  solution  of  linear  equations. 

Hare  we  point  otrtr^how  Winograd' s  method  can  speed  up  linear  pro¬ 
gramming  algorithms,  in  particular  the  revised  simplex  method, 

(see,  e.g.,  Simonnard  [2]).  1 


Winograd' s  algorithm:  * 

Let  X  -  (xn...,xn)My  -  (y1,...,yn>, 

[n/2]  * 
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[n/2] 

1>*  Z  y2J-ly2J 
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tn/2] 

Y  “  Zi  (x2j-l  +  y2J)(x2j  +  y2)-l*' 
j-1 

where  [t]  denotes  the  integer  part  of  t.  The  Inner  product  (x,y)  is 
then  given  by 


!V  -  5  *  T)»  if  n  is  even 

y-5-M+xy.ifnis 
n  n 


odd. 


In  the  sequel,  we  denote  (x,y)  simply  by  xy. 

Pricing  out  in  LPs: 

At  a  given  iteration,  let  tt  be  the  vector  of  simplex  multipliers 
and  let  J  be  the  set  of  nonbasic  columns  in  the  constraint  matrix.  A 
major  task  is  to  determine  which  column  should  enter  the  basis.  Let 
a^  stand  for  a  generic  column.  To  price  out,  we  need  to  compute 
na^,  j  €  J.  Assuming  that  there  are  m  constraints  and  k  columns  in  J, 
the  conventional  procedure  requires  km  multiplications  and  k(m  -  1) 
additions . 

Now  we  apply  Winograd's  method.  For  simplicity,  suppose  that  m 
is  even.  At  the  first  Iteration  only,  compute 

a/2 

Xj  *  L  *21- 1  *2i 

i-1 

for  all  j,  basic  or  not.  Of  course,  if  a^  is  a  unit  vector,  Xj  ■  0. 
(The  initial  basis  generally  consists  of  unit  vectors.)  Since  the  Xj's 
need  to  be  computed  only  once,  we  neglect  the  multiplications  and 
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additions  involved.  Thus,  pricing  out  with  Winograd's  scheme  requires 
(k  +  l)m/2  multiplications  and  k(3m/2  +  1)  +  m/2  -  1  additions  per 
iteration.  Typically,  k  »  m.  If  the  multiply  and  add  times  were 
identical,  the  two  procedures  would  be  roughly  equivalent.  If  the 
multiply  time  is  signigicantly  slower  than  the  add  time,  as  is  generally 
the  case,  the  acceleration  using  Winograd's  procedure  can  be  dramatic. 
Winograd's  procedure  requires  roughly  half  as  many  multiplications  as 
the  standard  procedure  and  roughly  50  per  cent  more  additions. 

Improvement  factor: 

If  p  is  the  fraction  of  time  spent  pricing  out  with  the  standard 
method  and  r  is  the  ratio  of  multiply  time  to  add  time,  then  using 
Winograd's  algorithm  is  approximately  2(r  +  l)/[p(r  +  3)  +  2(1  -  p)(r  +  1)] 
times  as  fast  as  using  a  standard  LP  code.  For  r  »  3,  a  fairly  repre¬ 
sentative  number  for  current  computers,  the  Improvement  factor  is 

8/(8  -  2p). 

2  2 
Now  we  estimate  p.  A  pivot  takes  about  m  multiplications  and  m 

additions.  By  comparison,  the  number  of  operations  to  find  the  repre¬ 
sentation  of  the  pivot  column  in  terms  of  the  current  basis  is  of  the  same 
order  and  the  number  to  determine  the  pivot  row  is  negligible.  We  do  not 
count  input/output  time.  Thus  p  is  about  k/(2m  ♦  k),  which  is  typically 
near  one.  For  r  ■  3,  we  can  expect  an  improvement  of  almost  25  percent. 

Other  LP  applications: 

We  do  not  see  where  Winograd's  procedure  would  be  helpful  in 
general  in  pivoting.  However,  sometimes  a  good  starting  basis  is  known 
(often,  in  Markov  renewal  programming  [l])  and  Winograd's  procedure  can 
then  be  applied  to  efficiently  compute  its  inverse.  See  [3]  for  details. 


Sparse  matrices: 

Various  programming  cricks  for  handling  sparse  matrices  are  avail¬ 
able.  When  using  these  tricks,  the  relative  efficiency  of  Wlnograd's 
procedure  should  be  reexamined.  We  believe  that  Wlnograd's  procedure 
will  generally  turn  out  to  be  faster. 
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